Phi fluctuates with surprisal: An empirical pre-study for the synthesis of the free energy principle and integrated information theory.

The Free Energy Principle (FEP) and Integrated Information Theory (IIT) are two ambitious theoretical approaches. The first aims to make a formal framework for describing self-organizing and life-like systems in general, and the second attempts a mathematical theory of conscious experience based on the intrinsic properties of a system. They are each concerned with complementary aspects of the properties of systems, one with life and behavior, the other with meaning and experience, so combining them has potential for scientific value. In this paper, we take a first step towards such a synthesis by expanding on the results of an earlier published evolutionary simulation study, which show a relationship between IIT-measures and fitness in differing complexities of tasks. We relate a basic information theoretic measure from the FEP, surprisal, to this result, finding that the surprisal of simulated agents' observations is inversely related to the general increase in fitness and integration over evolutionary time. Moreover, surprisal fluctuates together with IIT-based consciousness measures in within-trial time. This suggests that the consciousness measures used in IIT indirectly depend on the relation between the agent and the external world, and that it should therefore be possible to relate them to the theoretical concepts used in the FEP. Lastly, we suggest a future approach for investigating this relationship empirically.

Blocking D2/D3 dopamine receptors in male participants increases volatility of beliefs when learning to trust others.

The ability to learn about other people is crucial for human social functioning. Dopamine has been proposed to regulate the precision of beliefs, but direct behavioural evidence of this is lacking. In this study, we investigate how a high dose of the D2/D3 dopamine receptor antagonist sulpiride impacts learning about other people's prosocial attitudes in a repeated Trust game. Using a Bayesian model of belief updating, we show that in a sample of 76 male participants sulpiride increases the volatility of beliefs, which leads to higher precision weights on prediction errors. This effect is driven by participants with genetically conferred higher dopamine availability (Taq1a polymorphism) and remains even after controlling for working memory performance. Higher precision weights are reflected in higher reciprocal behaviour in the repeated Trust game but not in single-round Trust games. Our data provide evidence that the D2 receptors are pivotal in regulating prediction error-driven belief updating in a social context.

Effects of dopamine D2/3 and opioid receptor antagonism on the trade-off between model-based and model-free behaviour in healthy volunteers.

Human behaviour requires flexible arbitration between actions we do out of habit and actions that are directed towards a specific goal. Drugs that target opioid and dopamine receptors are notorious for inducing maladaptive habitual drug consumption; yet, how the opioidergic and dopaminergic neurotransmitter systems contribute to the arbitration between habitual and goal-directed behaviour is poorly understood. By combining pharmacological challenges with a well-established decision-making task and a novel computational model, we show that the administration of the dopamine D2/3 receptor antagonist amisulpride led to an increase in goal-directed or 'model-based' relative to habitual or 'model-free' behaviour, whereas the non-selective opioid receptor antagonist naltrexone had no appreciable effect. The effect of amisulpride on model-based/model-free behaviour did not scale with drug serum levels in the blood. Furthermore, participants with higher amisulpride serum levels showed higher explorative behaviour. These findings highlight the distinct functional contributions of dopamine and opioid receptors to goal-directed and habitual behaviour and support the notion that even small doses of amisulpride promote flexible application of cognitive control.

Conditioned Hallucinations and Prior Overweighting Are State-Sensitive Markers of Hallucination Susceptibility.

BACKGROUND: Recent advances in computational psychiatry have identified latent cognitive and perceptual states that predispose to psychotic symptoms. Behavioral data fit to Bayesian models have demonstrated an overreliance on priors (i.e., prior overweighting) during perception in select samples of individuals with hallucinations, corresponding to increased precision of prior expectations over incoming sensory evidence. However, the clinical utility of this observation depends on the extent to which it reflects static symptom risk or current symptom state. METHODS: To determine whether task performance and estimated prior weighting relate to specific elements of symptom expression, a large, heterogeneous, and deeply phenotyped sample of hallucinators (n = 249) and nonhallucinators (n = 209) performed the conditioned hallucination (CH) task. RESULTS: We found that CH rates predicted stable measures of hallucination status (i.e., peak frequency). However, CH rates were more sensitive to hallucination state (i.e., recent frequency), significantly correlating with recent hallucination severity and driven by heightened reliance on past experiences (priors). To further test the sensitivity of CH rate and prior weighting to symptom severity, a subset of participants with hallucinations (n = 40) performed a repeated-measures version of the CH task. Changes in both CH frequency and prior weighting varied with changes in auditory hallucination frequency on follow-up. CONCLUSIONS: These results indicate that CH rate and prior overweighting are state markers of hallucination status, potentially useful in tracking disease development and treatment response.

A generative framework for the study of delusions.

Despite the ubiquity of delusional information processing in psychopathology and everyday life, formal characterizations of such inferences are lacking. In this article, we propose a generative framework that entails a computational mechanism which, when implemented in a virtual agent and given new information, generates belief updates (i.e., inferences about the hidden causes of the information) that resemble those seen in individuals with delusions. We introduce a particular form of Dirichlet process mixture model with a sampling-based Bayesian inference algorithm. This procedure, depending on the setting of a single parameter, preferentially generates highly precise (i.e. over-fitting) explanations, which are compartmentalized and thus can co-exist despite being inconsistent with each other. Especially in ambiguous situations, this can provide the seed for delusional ideation. Further, we show by simulation how the excessive generation of such over-precise explanations leads to new information being integrated in a way that does not lead to a revision of established beliefs. In all configurations, whether delusional or not, the inference generated by our algorithm corresponds to Bayesian inference. Furthermore, the algorithm is fully compatible with hierarchical predictive coding. By virtue of these properties, the proposed model provides a basis for the empirical study and a step toward the characterization of the aberrant inferential processes underlying delusions.

Paranoia and belief updating during the COVID-19 crisis.

The COVID-19 pandemic has made the world seem less predictable. Such crises can lead people to feel that others are a threat. Here, we show that the initial phase of the pandemic in 2020 increased individuals' paranoia and made their belief updating more erratic. A proactive lockdown made people's belief updating less capricious. However, state-mandated mask-wearing increased paranoia and induced more erratic behaviour. This was most evident in states where adherence to mask-wearing rules was poor but where rule following is typically more common. Computational analyses of participant behaviour suggested that people with higher paranoia expected the task to be more unstable. People who were more paranoid endorsed conspiracies about mask-wearing and potential vaccines and the QAnon conspiracy theories. These beliefs were associated with erratic task behaviour and changed priors. Taken together, we found that real-world uncertainty increases paranoia and influences laboratory task behaviour.

Moral dilemmas and trust in leaders during a global health crisis.

Trust in leaders is central to citizen compliance with public policies. One potential determinant of trust is how leaders resolve conflicts between utilitarian and non-utilitarian ethical principles in moral dilemmas. Past research suggests that utilitarian responses to dilemmas can both erode and enhance trust in leaders: sacrificing some people to save many others ('instrumental harm') reduces trust, while maximizing the welfare of everyone equally ('impartial beneficence') may increase trust. In a multi-site experiment spanning 22 countries on six continents, participants (N = 23,929) completed self-report (N = 17,591) and behavioural (N = 12,638) measures of trust in leaders who endorsed utilitarian or non-utilitarian principles in dilemmas concerning the COVID-19 pandemic. Across both the self-report and behavioural measures, endorsement of instrumental harm decreased trust, while endorsement of impartial beneficence increased trust. These results show how support for different ethical principles can impact trust in leaders, and inform effective public communication during times of global crisis. PROTOCOL REGISTRATION STATEMENT: The Stage 1 protocol for this Registered Report was accepted in principle on 13 November 2020. The protocol, as accepted by the journal, can be found at https://doi.org/10.6084/m9.figshare.13247315.v1 .

TAPAS: An Open-Source Software Package for Translational Neuromodeling and Computational Psychiatry.

Psychiatry faces fundamental challenges with regard to mechanistically guided differential diagnosis, as well as prediction of clinical trajectories and treatment response of individual patients. This has motivated the genesis of two closely intertwined fields: (i) Translational Neuromodeling (TN), which develops "computational assays" for inferring patient-specific disease processes from neuroimaging, electrophysiological, and behavioral data; and (ii) Computational Psychiatry (CP), with the goal of incorporating computational assays into clinical decision making in everyday practice. In order to serve as objective and reliable tools for clinical routine, computational assays require end-to-end pipelines from raw data (input) to clinically useful information (output). While these are yet to be established in clinical practice, individual components of this general end-to-end pipeline are being developed and made openly available for community use. In this paper, we present the Translational Algorithms for Psychiatry-Advancing Science (TAPAS) software package, an open-source collection of building blocks for computational assays in psychiatry. Collectively, the tools in TAPAS presently cover several important aspects of the desired end-to-end pipeline, including: (i) tailored experimental designs and optimization of measurement strategy prior to data acquisition, (ii) quality control during data acquisition, and (iii) artifact correction, statistical inference, and clinical application after data acquisition. Here, we review the different tools within TAPAS and illustrate how these may help provide a deeper understanding of neural and cognitive mechanisms of disease, with the ultimate goal of establishing automatized pipelines for predictions about individual patients. We hope that the openly available tools in TAPAS will contribute to the further development of TN/CP and facilitate the translation of advances in computational neuroscience into clinically relevant computational assays.

Updating beliefs beyond the here-and-now: the counter-factual self in anosognosia for hemiplegia.

The syndrome of anosognosia for hemiplegia, or the lack of awareness for one's paralysis following right hemisphere stroke, can provide unique insights into the neurocognitive mechanisms of self-awareness. Yet it remains unclear whether anosognosia for hemiplegia is a modality-specific deficit of sensorimotor monitoring, or whether domain-general processes of attention and belief-updating converge to cause anosognosia for hemiplegia. Using a Bayesian learning framework, we formalized and empirically investigated the hypothesis that failures to update anosognosic beliefs can be explained by abnormalities in the relative uncertainty (i.e. precision) ascribed to prior beliefs versus sensory information in different contexts. We designed a new motor belief-updating task that manipulated both the temporal (prospective and retrospective) and spatial (hemispace most affected by inattention and hemispace less affected by inattention) conditions in which beliefs had to be updated, and we validated its sensitivity to anosognosia for hemiplegia in 26 patients with right hemisphere stroke. We then computed and empirically tested two different Bayesian predictors of prospective beliefs using two proxies for precision in anosognosia for hemiplegia patients: (i) standardized, neuropsychological measures of objective attention abilities, i.e. visuospatial neglect scores and (ii) subjective uncertainty reports, i.e. confidence ratings. Our results suggest that while neglect does not affect local, sensorimotor error monitoring, it does seem to affect the degree to which observed errors are used to update more general, prospective beliefs about counterfactual motor abilities in anosognosia for hemiplegia. Difficulties in such 'counterfactual' belief-updating were associated with disruptions in tracts of the ventral attentional network (i.e. superior longitudinal fasciculus connecting the temporo-parietal junction and ventral frontal cortex) and associated lesions to the insula, inferior parietal cortex and superior temporal regions. These results suggest that self-awareness extends beyond local, retrospective monitoring, requiring also salience-based, convergence of beliefs about the self that go beyond the 'here-and-now' of sensorimotor experience.

Paranoia and belief updating during a crisis.

The 2019 coronavirus (COVID-19) pandemic has made the world seem unpredictable. During such crises we can experience concerns that others might be against us, culminating perhaps in paranoid conspiracy theories. Here, we investigate paranoia and belief updating in an online sample (N=1,010) in the United States of America (U.S.A). We demonstrate the pandemic increased individuals' self-rated paranoia and rendered their task-based belief updating more erratic. Local lockdown and reopening policies, as well as culture more broadly, markedly influenced participants' belief-updating: an early and sustained lockdown rendered people's belief updating less capricious. Masks are clearly an effective public health measure against COVID-19. However, state-mandated mask wearing increased paranoia and induced more erratic behaviour. Remarkably, this was most evident in those states where adherence to mask wearing rules was poor but where rule following is typically more common. This paranoia may explain the lack of compliance with this simple and effective countermeasure. Computational analyses of participant behaviour suggested that people with higher paranoia expected the task to be more unstable, but at the same time predicted more rewards. In a follow-up study we found people who were more paranoid endorsed conspiracies about mask-wearing and potential vaccines - again, mask attitude and conspiratorial beliefs were associated with erratic task behaviour and changed priors. Future public health responses to the pandemic might leverage these observations, mollifying paranoia and increasing adherence by tempering people's expectations of other's behaviour, and the environment more broadly, and reinforcing compliance.

Cholinergic and dopaminergic effects on prediction error and uncertainty responses during sensory associative learning.

Navigating the physical world requires learning probabilistic associations between sensory events and their change in time (volatility). Bayesian accounts of this learning process rest on hierarchical prediction errors (PEs) that are weighted by estimates of uncertainty (or its inverse, precision). In a previous fMRI study we found that low-level precision-weighted PEs about visual outcomes (that update beliefs about associations) activated the putative dopaminergic midbrain; by contrast, precision-weighted PEs about cue-outcome associations (that update beliefs about volatility) activated the cholinergic basal forebrain. These findings suggested selective dopaminergic and cholinergic influences on precision-weighted PEs at different hierarchical levels. Here, we tested this hypothesis, repeating our fMRI study under pharmacological manipulations in healthy participants. Specifically, we performed two pharmacological fMRI studies with a between-subject double-blind placebo-controlled design: study 1 used antagonists of dopaminergic (amisulpride) and muscarinic (biperiden) receptors, study 2 used enhancing drugs of dopaminergic (levodopa) and cholinergic (galantamine) modulation. Pooled across all pharmacological conditions of study 1 and study 2, respectively, we found that low-level precision-weighted PEs activated the midbrain and high-level precision-weighted PEs the basal forebrain as in our previous study. However, we found pharmacological effects on brain activity associated with these computational quantities only when splitting the precision-weighted PEs into their PE and precision components: in a brainstem region putatively containing cholinergic (pedunculopontine and laterodorsal tegmental) nuclei, biperiden (compared to placebo) enhanced low-level PE responses and attenuated high-level PE activity, while amisulpride reduced high-level PE responses. Additionally, in the putative dopaminergic midbrain, galantamine compared to placebo enhanced low-level PE responses (in a body-weight dependent manner) and amisulpride enhanced high-level precision activity. Task behaviour was not affected by any of the drugs. These results do not support our hypothesis of a clear-cut dichotomy between different hierarchical inference levels and neurotransmitter systems, but suggest a more complex interaction between these neuromodulatory systems and hierarchical Bayesian quantities. However, our present results may have been affected by confounds inherent to pharmacological fMRI. We discuss these confounds and outline improved experimental tests for the future.

Aberrant computational mechanisms of social learning and decision-making in schizophrenia and borderline personality disorder.

Psychiatric disorders are ubiquitously characterized by debilitating social impairments. These difficulties are thought to emerge from aberrant social inference. In order to elucidate the underlying computational mechanisms, patients diagnosed with major depressive disorder (N = 29), schizophrenia (N = 31), and borderline personality disorder (N = 31) as well as healthy controls (N = 34) performed a probabilistic reward learning task in which participants could learn from social and non-social information. Patients with schizophrenia and borderline personality disorder performed more poorly on the task than healthy controls and patients with major depressive disorder. Broken down by domain, borderline personality disorder patients performed better in the social compared to the non-social domain. In contrast, controls and major depressive disorder patients showed the opposite pattern and schizophrenia patients showed no difference between domains. In effect, borderline personality disorder patients gave up a possible overall performance advantage by concentrating their learning in the social at the expense of the non-social domain. We used computational modeling to assess learning and decision-making parameters estimated for each participant from their behavior. This enabled additional insights into the underlying learning and decision-making mechanisms. Patients with borderline personality disorder showed slower learning from social and non-social information and an exaggerated sensitivity to changes in environmental volatility, both in the non-social and the social domain, but more so in the latter. Regarding decision-making the modeling revealed that compared to controls and major depression patients, patients with borderline personality disorder and schizophrenia showed a stronger reliance on social relative to non-social information when making choices. Depressed patients did not differ significantly from controls in this respect. Overall, our results are consistent with the notion of a general interpersonal hypersensitivity in borderline personality disorder and schizophrenia based on a shared computational mechanism characterized by an over-reliance on beliefs about others in making decisions and by an exaggerated need to make sense of others during learning specifically in borderline personality disorder.

Hierarchical Bayesian models of social inference for probing persecutory delusional ideation.

While persecutory delusions (PDs) have been linked to fallacies of reasoning and social inference, computational characterizations of delusional tendencies are rare. Here, we examined 151 individuals from the general population on opposite ends of the PD spectrum (Paranoia Checklist [PCL]). Participants made trial-wise predictions in a probabilistic lottery, guided by advice from a more informed human and a nonsocial cue. Additionally, 2 frames differentially emphasized causes of invalid advice: (a) the adviser's possible intentions (dispositional frame) or (b) the rules of the game (situational frame). We applied computational modeling to examine possible reasons for group differences in behavior. Comparing different models, we found that a hierarchical Bayesian model (hierarchical Gaussian filter) explained participants' responses better than other learning models. Model parameters determining participants' belief updates about the adviser's fidelity and the contribution of prior beliefs about fidelity to trial-wise decisions, respectively, showed significant Group × Frame interactions: High PCL scorers held more rigid beliefs about the adviser's fidelity across both experimental frames and relied less on advice in situational frames than low scorers. These results suggest that PD tendencies are associated with rigid beliefs and prevent adaptive use of social information in "safe" contexts. This supports previous proposals of a link between PD and aberrant social inference. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Neural arbitration between social and individual learning systems.

Decision making requires integrating knowledge gathered from personal experiences with advice from others. The neural underpinnings of the process of arbitrating between information sources has not been fully elucidated. In this study, we formalized arbitration as the relative precision of predictions, afforded by each learning system, using hierarchical Bayesian modeling. In a probabilistic learning task, participants predicted the outcome of a lottery using recommendations from a more informed advisor and/or self-sampled outcomes. Decision confidence, as measured by the number of points participants wagered on their predictions, varied with our definition of arbitration as a ratio of precisions. Functional neuroimaging demonstrated that arbitration signals were independent of decision confidence and involved modality-specific brain regions. Arbitrating in favor of self-gathered information activated the dorsolateral prefrontal cortex and the midbrain, whereas arbitrating in favor of social information engaged the ventromedial prefrontal cortex and the amygdala. These findings indicate that relative precision captures arbitration between social and individual learning systems at both behavioral and neural levels.

Bayesian modelling captures inter-individual differences in social belief computations in the putamen and insula.

Computational models of social learning and decision-making provide mechanistic tools to investigate the neural mechanisms that are involved in understanding other people. While most studies employ explicit instructions to learn from social cues, everyday life is characterized by the spontaneous use of such signals (e.g., the gaze of others) to infer on internal states such as intentions. To investigate the neural mechanisms of the impact of gaze cues on learning and decision-making, we acquired behavioural and fMRI data from 50 participants performing a probabilistic task, in which cards with varying winning probabilities had to be chosen. In addition, the task included a computer-generated face that gazed towards one of these cards providing implicit advice. Participants' individual belief trajectories were inferred using a hierarchical Gaussian filter (HGF) and used as predictors in a linear model of neuronal activation. During learning, social prediction errors were correlated with activity in inferior frontal gyrus and insula. During decision-making, the belief about the accuracy of the social cue was correlated with activity in inferior temporal gyrus, putamen and pallidum while the putamen and insula showed activity as a function of individual differences in weighting the social cue during decision-making. Our findings demonstrate that model-based fMRI can give insight into the behavioural and neural aspects of spontaneous social cue integration in learning and decision-making. They provide evidence for a mechanistic involvement of specific components of the basal ganglia in subserving these processes.

Ketamine Affects Prediction Errors about Statistical Regularities: A Computational Single-Trial Analysis of the Mismatch Negativity.

The auditory mismatch negativity (MMN) is significantly reduced in schizophrenia. Notably, a similar MMN reduction can be achieved with NMDA receptor (NMDAR) antagonists. Both phenomena have been interpreted as reflecting an impairment of predictive coding or, more generally, the "Bayesian brain" notion that the brain continuously updates a hierarchical model to infer the causes of its sensory inputs. Specifically, neurobiological interpretations of predictive coding view perceptual inference as an NMDAR-dependent process of minimizing hierarchical precision-weighted prediction errors (PEs), and disturbances of this putative process play a key role in hierarchical Bayesian theories of schizophrenia. Here, we provide empirical evidence for this theory, demonstrating the existence of multiple, hierarchically related PEs in a "roving MMN" paradigm. We applied a hierarchical Bayesian model to single-trial EEG data from healthy human volunteers of either sex who received the NMDAR antagonist S-ketamine in a placebo-controlled, double-blind, within-subject fashion. Using an unrestricted analysis of the entire time-sensor space, our trial-by-trial analysis indicated that low-level PEs (about stimulus transitions) are expressed early (102-207 ms poststimulus), while high-level PEs (about transition probability) are reflected by later components (152-199 and 215-277 ms) of single-trial responses. Furthermore, we find that ketamine significantly diminished the expression of high-level PE responses, implying that NMDAR antagonism disrupts the inference on abstract statistical regularities. Our findings suggest that NMDAR dysfunction impairs hierarchical Bayesian inference about the world's statistical structure. Beyond the relevance of this finding for schizophrenia, our results illustrate the potential of computational single-trial analyses for assessing potential pathophysiological mechanisms.

Paranoia as a deficit in non-social belief updating.

Paranoia is the belief that harm is intended by others. It may arise from selective pressures to infer and avoid social threats, particularly in ambiguous or changing circumstances. We propose that uncertainty may be sufficient to elicit learning differences in paranoid individuals, without social threat. We used reversal learning behavior and computational modeling to estimate belief updating across individuals with and without mental illness, online participants, and rats chronically exposed to methamphetamine, an elicitor of paranoia in humans. Paranoia is associated with a stronger prior on volatility, accompanied by elevated sensitivity to perceived changes in the task environment. Methamphetamine exposure in rats recapitulates this impaired uncertainty-driven belief updating and rigid anticipation of a volatile environment. Our work provides evidence of fundamental, domain-general learning differences in paranoid individuals. This paradigm enables further assessment of the interplay between uncertainty and belief-updating across individuals and species.

Everyone has had fleeting concerns that others might be against them at some point in their lives. Sometimes these concerns can escalate into paranoia and become debilitating. Paranoia is a common symptom in serious mental illnesses like schizophrenia. It can cause extreme distress and is linked with an increased risk of violence towards oneself or others. Understanding what happens in the brains of people experiencing paranoia might lead to better ways to treat or manage it. Some experts argue that paranoia is caused by errors in the way people assess social situations. An alternative idea is that paranoia stems from the way the brain forms and updates beliefs about the world. Now, Reed et al. show that both people with paranoia and rats exposed to a paranoia-inducing substance expect the world will change frequently, change their minds often, and have a harder time learning in response to changing circumstances. In the experiments, human volunteers with and without psychiatric disorders played a game where the best choices change. Then, the participants completed a survey to assess their level of paranoia. People with higher levels of paranoia predicted more changes would occur and made less predictable choices. In a second set of experiments, rats were put in a cage with three holes where they sometimes received sugar rewards. Some of the rats received methamphetamine, a drug that causes paranoia in humans. Rats given the drug also expected the location of the sugar reward would change often. The drugged animals had harder time learning and adapting to changing circumstances. The experiments suggest that brain processes found in both rats, which are less social than humans, and humans contribute to paranoia. This suggests paranoia may make it harder to update beliefs. This may help scientists understand what causes paranoia and develop therapies or drugs that can reduce paranoia. This information may also help scientists understand why during societal crises like wars or natural disasters humans are prone to believing conspiracies. This is particularly important now as the world grapples with climate change and a global pandemic. Reed et al. note paranoia may impede the coordination of collaborative solutions to these challenging situations.

Atypical processing of uncertainty in individuals at risk for psychosis.

Current theories of psychosis highlight the role of abnormal learning signals, i.e., prediction errors (PEs) and uncertainty, in the formation of delusional beliefs. We employed computational analyses of behaviour and functional magnetic resonance imaging (fMRI) to examine whether such abnormalities are evident in clinical high risk (CHR) individuals. Non-medicated CHR individuals (n = 13) and control participants (n = 13) performed a probabilistic learning paradigm during fMRI data acquisition. We used a hierarchical Bayesian model to infer subject-specific computations from behaviour - with a focus on PEs and uncertainty (or its inverse, precision) at different levels, including environmental 'volatility' - and used these computational quantities for analyses of fMRI data. Computational modelling of CHR individuals' behaviour indicated volatility estimates converged to significantly higher levels than in controls. Model-based fMRI demonstrated increased activity in prefrontal and insular regions of CHR individuals in response to precision-weighted low-level outcome PEs, while activations of prefrontal, orbitofrontal and anterior insula cortex by higher-level PEs (that serve to update volatility estimates) were reduced. Additionally, prefrontal cortical activity in response to outcome PEs in CHR was negatively associated with clinical measures of global functioning. Our results suggest a multi-faceted learning abnormality in CHR individuals under conditions of environmental uncertainty, comprising higher levels of volatility estimates combined with reduced cortical activation, and abnormally high activations in prefrontal and insular areas by precision-weighted outcome PEs. This atypical representation of high- and low-level learning signals might reflect a predisposition to delusion formation.

Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models.

Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans.